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Abstract. We prove that if an area-preserving homeomorphism of the torus 
in the homotopy class of the identity has a rotation set which is a nonde- 
generate vertical segment containing the origin, then there exists an essential 
invariant annulus. In particular, some lift to the universal covering has uni- 
formly bounded displacement in the horizontal direction. 



1. Introduction 

Let T 2 = R 2 /Z 2 denote the two-dimensional torus, and ir: M 2 — > Z 2 the canonical 
projection. Consider a homeomorphism / : T 2 — > T 2 homotopic to the identity, and 
a lift /: M 2 — > M 2 of /. The rotation set p(f) of /, introduced by Misiurewicz and 
Ziemian in [MZ 89 as a generalization of the rotation number of an orientation- 
preserving circle homeomorphism, is defined as the set of all limits of sequences of 
the form 

(1) v= lim (f n "(zk)-z k )/n k , 

k— S-OO 

where (nk)kefi is a sequence of integers such that n k — > oo as k — > oo. Roughly 
speaking, this set measures the average asymptotic rotation of orbits. It is known 
that p{f) is always compact and convex, and if v £ /?(/), is extremal or interior in 
p(f), then there is z £ K 2 such that the pointwise rotation vector 

(2) p(f,z) = lim (f n {z)-z)/n 

n— >oo 

exists and coincides with v. 

In this article, we consider area-preserving homeomorphisms, or more generally 
homeomorphisms preserving a Borel probability measure of full support. The ques- 
tion that we address is how the unbounded behavior of orbits along a given direction 
in the universal covering affects the rotation set. In principle, the rotation set does 
not distinguish fixed points from orbits that become unbounded at a sublinear rate; 
but does this kind of phenomenon exist? 

An example given in KT 1 2a shows that the answer is positive in general: there 
is an ergodic area-preserving C°° diffeomorphism with a lift whose rotation set is 
{(0, 0)}, but such that almost every point has an unbounded orbit in every direction 
(moreover, the orbit of almost every point visits every fundamental domain in the 
universal covering). However, in |KT12bj it is shown that this type of example is 
very particular, as it forces the existence of a large "essential" continuum of fixed 
points. There are also examples where the rotation set consists of a single vector 
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w ^ Q 2 , but there are orbits which are unbounded in the direction perpendicular 
to w |Jaj09l |KK09]. 

When the rotation set is not a single point, the situation turns out to be rather 
different. This is the case addressed in this article. The general idea behind our 
main result is illustrated by the theory of (area-preserving) twist maps. For twist 
maps of the cylinder T 1 x E, Mather showed in |Mat91| that either there exists a 
"barrier" (in this setting, an essential invariant circle) which impedes the drift of 
orbits in the vertical direction, or there is an orbit which converges towards infinity 
(moreover, the orbit converges toward the upper end in the future and the lower end 
in the past, or vice- versa). For twist maps on T 2 , the later result was improved in 
[Sli02j : the non-existence of a "barrier" implies the existence of orbits with nonzero 
average speed of rotation in the direction perpendicular to the direction of twist 
(i.e. with nonzero shear rotation number). 

The result from |Sli02] was generalized in |AGTllj . where the technical and 
restrictive twist condition was replaced by the condition that the homeomorphism 
of T 2 be in a Dehn homotopy class (which implies a weak type of topological twist). 

Our main result considers the case of area-preserving homeomorphisms of T 2 
homotopic to the identity. The twist condition is replaced by a weak condition on 
the existence of orbits with different average rotation speed (i.e. rotation vectors). 

To be more precise, let us say that / has bounded displacement in the direction 
v € M 2 at a point z £ M 2 if there exists M > such that 

\Pv{f n (z) -z)\< M, for all n e Z, 

where p v (x) = (x; vj ||u||) is the projection onto the direction v. If the number M 
can be chosen independent of z € R 2 , then we say that / has uniformly bounded 
displacement in the direction v. 

By an essential annulus we mean a subset A of T 2 homcomorphic to an open 
annulus and containing a homotopically nontrivial loop 7 of T 2 . The homological 
direction of A is the direction of the line M.v, where wgZ 2 ~ i?i(T 2 ,Z) represents 
the homology class of 7. In particular, a vertical annulus is one with homological 
direction {0} x M. The main result of this article is the following: 

Theorem A. Let / : T 2 — > T 2 be a homeomorphism homotopic to the identity 
preserving a Borel probability measure of full support, and suppose that some lift 
/: K 2 — > R 2 has a rotation set p(f) = {0} x [a, b], where a < b. Then f has 
uniformly bounded displacement in the horizontal direction. Moreover, there exists 
an essential invariant annulus (which is necessarily vertical). 

A similar result was obtained by Davalos in |Davllj under the assumption that 
there is a rational point in the rotation set which is not realized by a periodic orbit; 
however this hypothesis is disjoint from ours, since the existence of an invariant 
measure of full support already guarantees the realization of all rational points by 
periodic orbits (see Theorem 2.3). In |BT12j . a vesion of Theorem [A| was obtained 



under strong additional hypotheses (namely that the measure of full support be 
ergodic and with a non-rational rotation vector). 

When / has some periodic essential annulus and p(f) is not a single point, it is 
easy to verify that /?(/) is a segment of rational slope (parallel to the homological 
direction of the periodic annulus) containing some point of rational coordinates. As 
a consequence of Theorem [3] the converse of this fact holds: 
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Theorem B. Let /: T 2 — > T 2 be a homeomorphism homotopic to the identity 
preserving a Borel probability measure of full support, and f a lift of f such that 
p{f) has more than one point. Then f has a periodic essential annulus if and only 
if p(f) is a segment of rational slope containing some point of rational coordinates 
(and in this case the homological direction of the annulus is parallel to the rotation 
segment). 

Note that Theorem [A] says that the dynamics of / is essentially "annular"; the 
horizontal homological direction of T 2 plays no role in the dynamics. In particular, 
Theorem [X] allows to translate existing results about rotation sets for homeomor- 
phisms of the annulus (e.g. |Han 90] ) to this setting 

A simple reformulation of Theorem [5] also leads to the following 

Corollary C. Let /: T 2 — > T 2 be a homeomorphism homotopic to the identity, 
preserving a Borel probability measure of full support, and f: M 2 — > M 2 a lift of f. 
Suppose that p{f) is not a single point, and for some nonzero »eZ 2 , 



sup \p v (f n (z)-z)\ 

zGR 2 ,n6Z 



then p„ (/£>(/)) ^ {0}. 



It seems reasonable to expect that, under the hypotheses of the previous theorem, 
sup p„ (/?(/)) > whenever sup zgR 2 n &Pv(f n i. z ) — z) > 0. The main difficulty to 
conclude the latter fact is the following question: suppose that p(f) has nonempty 
interior and is contained in {(x,y) : x < 0}. If the boundary of p(f) contains a 
vertical segment through the origin, does it follow that the horizontal displacement 
of orbits is uniformly bounded above? The arguments from the present article 
fail in this setting mainly due to the fact that the dynamics lifted to a horizontal 



cylinder (as done in E 3.4 ) need not be nonwandering under these hypotheses. 

This article is organized as follows. Section 2 introduces some notation, termi- 
nology and preliminary results. In particular, it includes a consequence of |KLN12j 
about the possible rotation numbers of periodic points in the boundary of an in- 
variant open topological disk, which is fundamental to obtain the uniformity in 
Theorem |A"| Section 3 contains the proof of the uniform boundedness of deviations 
in the horizontal direction under the hypotheses of Theorem [A] In Section 4, a gen- 
eral result about the realization of rational rotation vectors by periodic points in 
certain invariant continua is given. This is used in Section 5 to prove the existence 
of an invariant vertical annulus for a nonwandering homeomorphism whenever the 
horizontal displacement is uniformly bounded and the rotation set is not a point. 
Finally, in Section 6 the main theorems are proved using the preceding results. 



2. Notation and preliminaries 

The sets E*, M. 2 , Z* and Z 2 denote the set of all non-zero elements of the corre- 
sponding spaces, e.g. Z 2 = {v e Z 2 : v ^ (0, 0)} and similarly for the other spaces. 
We denote by Pi and P2 the projections onto the first and second coordinates of 
K 2 , respectively. 

Given u, v E E 2 , we denote by (u;v) their Euclidean scalar product. The or- 
thogonal projection onto the direction of v is denoted by 

p v (z) = (z;v/\\v\\). 
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From now on we assume that /: T 2 — > T 2 is a homeomorphism nomotopic to 
the identity. 

2.1. The rotation set. . 

The rotation set, as defined in the introduction, satisfies the following properties 
(see |MZ89j and |KK08j Lemma 4.2]) 

(1) p(f n + v) = np(f) + v for any n £ Z and v £ Z 2 ; 

(2) piAfA- 1 ) = Ap(f) for any A G GL(2,Z). 

In particular, if p(f) is a segment of rational slope, one may always find A £ 
GL(2,Z) such that p(AfA^ 1 ) is a vertical segment (see [KK081 Remark 2.5]). Note 
that AfA^ 1 is a lift to K 2 of a homeomorphism of T 2 homotopic to the identity 
and conjugate to / (via the map Aj2 induced by A on T 2 ). 

2.2. Essential and inessential sets. An open subset U of T 2 is said to be inessen- 
tial if every loop in U is homotopically trivial in T 2 ; otherwise, U is essential. An 
arbitrary set E is called inessential if it has some inessential neighborhood. We say 
that E is fully essential if T 2 \ E is inessential. 

2.3. The sets U' t {z). Given z £ T 2 and e > 0, denote by U' e {z,f) (or simply 
U' e (z) when there is no ambiguity) the connected component of \J neZ f n (B € (z)) 
containing z. Suppose that f n (B e (z)) intersects B e (z) for some n £ N (otherwise, 
U' e (z) = B t {z)). Since / permutes the connected components of Unez f n (Be(z)), it 
follows that U' e (z) — f n (U' e (z)), and if n £ N is chosen minimal with that property, 
then f k (U' e (z)) is disjoint from U' e {z)) whenever 1 < k < n. In particular, if n > 1 
then U' e {z) is disjoint from its image. 

2.4. Essential and inessential points. Following [ KT12c| . we say that x £ T 2 is 
inessential if there exists e > such that the set U' t (x) is inessential. Otherwise, we 
say that x is essential. The set of inessential points is an open invariant set denoted 
by Ine(/). Its complement is the set of essential points, denoted Ess(/), which is a 
closed invariant set. 

2.5. Annular and strictly toral homeomorphisms. Recall from |KT12c| that 
a nonwandering homeomorphism / is called annular if there is v £ Z 2 and a lift 
/ of / such that / has uniformly bounded deviations in the direction v; and / is 
called strictly toral if Fix(/ fe ) is not fully essential and f k is non-annular for each 
k £ N. The next result is contained in Theorem A and part (4) of Proposition 1.4 
of |KT12cj . 

Theorem 2.1 f |KT12c] l. If f is strictly toral, then Ine(/) is inessential, and 
Ess(/) is fully essential. Furthermore, if x £ Ess(/) then U' e (x) is fully essential 
for any e > 0. 

We will also need the next result, which is Theorem D of [K T12cj . 

Theorem 2.2 ( KT12c ). Let f be a strictly toral homeomorphism with a lift f, and 
suppose that for some v £ Z 2 and g£N there exists z £ M 2 such that f q (z) = z + v. 
Then there exists z' £ M 2 such that f q {z r ) = z' + v and tt(z') £ Ess(/). 
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2.6. Realization of rational rotation vectors. The previous theorem is useful 
in combination with the following 

Theorem 2.3. Suppose that f preserves a Borel probability measure of full support 
on T 2 , and let f be a lift of f to K 2 . If p(f) is an interval and v/q £ p(f) for some 
v £ Z 2 and q £ N, then there exists z £ K 2 such that f q (z) = z + v. 

This was proved by Franks for the case of area-preserving homeomorphisms 
|Fra95j . In [KK08 the authors prove a version of the same theorem replacing 
the preservation of area by the curve intersection property (i.e. the property that 
any essential loop intersects its image by /). The latter version easily implies 
that Franks' theorem is still valid if one replaces the area-preserving hypothesis by 
the preservation of any Borel probability measure of full support, as stated here. 
Moreover, a more recent result of Davalos |Davll] implies that the same result is 
true if / is either nonwandering or non-annular. 

2.7. Nonwandering homeomorphisms. We will use the following facts (for the 
first one, see |KorlO( Remark 4.1]). 

Proposition 2.4. If f is a nonwandering homeomorphism of a topological space, 
then so is /" . 

Proposition 2.5. Let f : S — >• S be a nonwandering homeomorphism of a surface, 
and f ': S — > S a lift of f by a finite covering w: S — > S. Then f is nonwandering. 

Proof. Let U\ C S be a nonempty open set. Since / is nonwandering, there exists 
ni £ N and a deck transformation T\ £ Deck(Tr) such that U 2 ■= UiC\Ti(f ni (Ui)) ^ 
0. The fact that / is nonwandering also implies that there is n 2 > tlx and 
T 2 G Deck(?f) such that U 2 n T 2 (f n2 (U 2 )) ^ 0. Repeating this process we ob- 
tain recursively an increasing sequence (ni),gN of integers, a sequence (Tj)^ e ^ of 
Deck transformations, and a decreasing sequence ({7j)igN of nonempty open sets, 
such that 

Ui+i — UinTi(f H (Ui)). 

Note that, for any k £ N, 

k k 

u k+1 c n Tiirm) c n Ti(f ni (Ui)). 

i=l i=l 

Let k be larger than the number of elements of Deck(7r). Then there exist 1 < 
i < j < k such that = Tj. This implies that f ni {U\) intersects f nj (Ui), and so 
f nj ~ ni (Ui) n U\ ^ 0. This proves that / is nonwnandering. □ 

2.8. A result on prime ends rotation numbers. We also need the following 
result, which is Theorem B of [KLN12 (stated in a simplified version). 

Theorem 2.6 ([KLN12 ). Let f : ]R 2 — > R 2 be an orientation preserving homeomor- 
phism and U C K 2 be an open f -invariant topological disk. If f is nonwandering 
in U and the prime ends rotation number of f in U is not zero ( mod 7L ), then there 
are no fixed points of f on the boundary of U . Moreover, if U is unbounded, then 
there are no fixed points of f in the complement of U . 
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For the sake of brevity of the exposition, we will not enter into the details of prime 
ends, since we will use them only tangentially. We only mention that the prime ends 
compactification of the open topological disk U C R 2 is obtained by attaching a 
circle called the circle of prime ends of U, thus obtaining a space [/LIT 1 topologized 
in a way that it is homeomorphic to the closed unit disk. If / is a homeomorphism 
of R 2 leaving U invariant, then f\jj extends to U UT 1 . The prime ends rotation 
number of / in U, denoted p(f, U) 6 R/Z, is the usual Poincare rotation number of 
the orientation preserving homeomorphism induced on T 1 by the extension of f\u ■ 
For more details and definitions, we refer the reader to |KLN12j . 



We will use a consequence of Theorem 2.6 for which we need a definition. Let 
/ be an orientation preserving homeomorphism of R 2 leaving U invariant and such 
that f\u is nonwandering. The latter condition implies that f\u has some fixed 
point Zo, due to a classic result of Brouwer |Fat87j . Consider the annulus R 2 \ {zq}, 
which we may identify with A = T 1 x R via a homeomorphism. We can regard / as 
a homeomorphism of AU{zo} — R 2 , where zq is the lower end of A. Let 7Ti : R 2 — > A 
be the covering projection, and /: R 2 — > A a lift of Since f\& is isotopic to 
the identity, / commutes with the Deck transformations (x, y) H> (x + k,y), fee Z. 
If z 7^ zq is a periodic point of /, we may define its rotation number associated 
to the lift / as follows: let z S n^ 1 (z) be any point. Then there exist k € Z and 
m € N such that f m (z) = z + (k,0), and these numbers do not depend on the 
choice of z. The rotation number p(f,z) is then defined to be k/m. Note that 
p(f + (k,0),z)=p(f,z) + (k,0). 

Corollary 2.7. All periodic points in dU have the same rotation number associated 
to f . If U is unbounded, the same is true for all periodic points in R 2 \ U. 

Proof. Suppose for a contradiction that / has two periodic points in d U with 
different rotation numbers associated to a lift /; i.e. there are xo,xi <G R 2 such 
that n^Xi) € dU and f mi {xi) = x t + (k u 0), with k /m ^ fei/mi. Using f m ° mi 



instead of / (which is also nonwandering in U by Proposition 2.4 1, we may assume 
that mi — m 2 = 1. Moreover, we may assume that k = by choosing the lift / 
appropriately. Thus, xq is a fixed point of / and f(xi) =xi + {k\, 0) with k\ ^ 0. 

Let A = (R/2fciZ) x R, and r: R 2 — s- A the covering projection. Note that 
7Ti : R 2 — > A induces on A a finite covering tti : A — > A, whose group of Deck 
transformations is generated by the map (x, y) H> (x + l+R/2fciZ, y). In particular, 
the homeomorphism T(x, y) — {x + k\ + R/2fciZ, y) on A is a Deck transformation 
of tti such that T 2 = Id. 

The map / induces on A a homeomorphism /, which is a lift of /|a by tti- Since 
f\ a is nomotopic to the identity, / commutes with the covering transformations 
and in particular with T. Denote by A* = A U {zq} where Zq is the lower end of A, 
topologized in the usual way so that A* ~ R 2 . We can extend / and T and regard 
them as maps of A* by fixing Zq . 

Let U = 7r _1 ([/ \ zq) U {z^}. Then U is an open /-invariant and T-invariant 
topological disk, and U is unbounded in A* if and only if U is unbounded in R 2 . 
The fact that f\u\{z } is nonwandering implies that /|{y\{ z *} is also nonwandering 
due to Proposition |2.5| 
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Since T 2 is the identity and T has no fixed points, one easily sees that the map 
induced on the prime ends of U by T has a rotation number 1/2 (mod Z) (for 
instance, any accessible prime end of U is periodic of period 2 and non-fixed by the 
map induced by T 2 on the circle of prime ends) . 

Since / commutes with T, the respective maps induced on the prime ends of U 
also commute. It is an easy consequence of the definition of rotation number that, 
if two orientation-preserving circle homeomorphisms commute, then the rotation 
number of their composition is the sum of their rotation numbers (mod Z). Thus, 

p(f /, U) = p(l U) + 1/2 (mod Z). 

In particular, if p(f, U) = 0, then p(ff, U) = 1/2 ^ (mod Z). Note that Tf is 
another lift of /|a to A, and therefore is also nonwandering in U. Let xq = t(xq) and 
x\ = t(xi). Then xq is a fixed point of /, and x\ is a fixed point of Tf = T _1 / '. 
Moreover, both xq and x\ belong to d U. Thus both maps / and Tf have a 
fixed point in d U, but one of them has a nonzero prime ends rotation number, 
contradicting Theorem |2.6| 

This proves the first claim of the theorem. In the case that U is unbounded, the 
same proof by contradiction applies, choosing Xi in R 2 \ U instead of d U. □ 

3. Proof of Theorem E uniformly bounded deviations 

We will divide the proof of Theorem [5] in two parts. This section is devoted 
to the proof of the first part, which is contained in the following proposition. The 
proof of the existence of an essential invariant annulus is postponed to Section [5j 

Proposition 3.1. Under the hypotheses of Theorem^A^ there exists M > such 
that 

|Pi(/™(z) - z)\ < M for all z e M 2 , n G Z. 

The remainder of this section contains the proof of this proposition. 

Assume that / is under the hypotheses of Theorem |Aj i.e. / preserves a Borel 
probability measure p of full support and p(f) is a vertical nondegenerate interval 
containing the origin. 

Claim 1. We may assume that {0} x [—1, 1] c p(f). 

Proof. Note that p(f h - v) = kp(f) - v for any k £ Z and v £ Z 2 (see |MZ89j ). 
and so letting g = f^ — (0,1) for appropriately chosen integers k, I we have that 
{0} x [—1, 1] c p{g), and g is a lift of f k . It is easy to verify that g has uniformly 
bounded deviations in the horizontal direction if and only if so does /. Thus we 
may use f k and g in place of / and /. □ 

Claim 2. There exist points zq, z\ in K 2 such that f(zi) = zi + (0,i) for 
i£ {-1,0,1} 

Proof. It follows from Theorem |2.3| and from our previous assumption. □ 



The proof of Proposition 3.1 will be by contradiction. Thus, from now on we 
assume that / has unbounded deviations in the horizontal direction, i.e. 

sup{|Pi(/"(z) -z)\:z£ M 2 , n £ Z} = oo. 

Under this assumption, we have 
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Claim 3. f is strictly toral. 

Proof. Let us first show that f k is non-annular for any k € N. By Theorem 2.3 and 
the fact that p(f) contains the origin, we know that / has a fixed point. Thus, if 
f k is annular for some k £ N, then so is / (by |KT12c[ Proposition 1.4(5)]). The 
fact that p{f) is a nondegenerate vertical interval easily implies that / can only be 
annular if / has uniformly bounded deviations in the horizontal direction, which 
by our assumptions is not possible. 

Now suppose that Fix(/ fc ) is fully essential for some k € N. Since f k is non- 
annular, by |KT12c| Proposition 5.1] applied to f k , some lift (and thus any lift) 
of f k to M 2 has a rotation set consisting of a single point. This is a contradiction, 
since p(f ) = kp(f) is a nondegenerate interval. 

Thus Fix(/ fc ) is not fully essential, and f k is non-annular for any k G N, which 
means that / is strictly toral. □ 

Claim 4. We may assume that Tr(zi) € Ess(J) for each i € { — 1,0, 1}. 

Proof. It follows from Theorem |2.2| □ 



3.1. The sets oj + and The definitions and properties that we introduce here 
were already used in [Tall 2 , K T12bj . Since we work with the vertical direction as 
a reference, we use a simplified notation. 

Denote by H + and H~ the half-planes {(x,y) : x > 0} and {(x,y) : x < 0}, 
respectively. Let / : T 2 — > T 2 be a homeomorphism homotopic to the identity, and 
/: K 2 — > K 2 a lift of /. Define the set uj+ C M 2 as the union of all unbounded 
connected components of 

oo 
i— — oo 

The set w_ is defined analogously using H~ instead of H + . 

The following properties are easy consequences of the definitions. 

Proposition 3.2. The sets u± are closed, and 

• /(w±) = w ± ; 

• cj ± + (0, b) C uj± for all b € Z; 

• lu + + (a, 0) C w+ and w_ — (a, 0) C UJ- for all a € N; 

• cj-t is non- separating, and its complement is simply connected. 

• Every connected component of w+ is unbounded to the right, and every 
connected component o/w_ is unbounded to the left. 

Claim 5. w + and oj- are non-empty. 

Proof. Since the origin lies in the boundary of the rotation set, this is a direct 
consequence of Lemma 4 and Corollary 1 of [Talfjj . □ 

The following property holds whenever / is strictly toral and the sets cj_ and 
U+ are nonempty: 

Proposition 3.3. Ess(/) C 7r(w + ) C\tt(w-) 



Proof. If z G Ess(/), then by Theorem 2.1 we know that U' e (z) is fully essential for 
any e > 0. This implies that every connected component of 7r _1 (T 2 \ U' e (z)) = K 2 \ 
7r _1 (J7 e '(z)) is bounded |KT12c[ Proposition 1.3]. Since lu + is nonempty and each 
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connected component of w+ is unbounded, it follows that lu + n n (U' e (z)) ^ 0. In 
other words, 7r(w + ) intersects U' e (z). Since tt(uj + ) is invariant, the definition of U' e {z) 
implies that 7r(w+) intersects B e (z). Since this holds for any e > 0, we conclude 
that z G 7r(cj + ). The same argument applied to u;_ in place of w+ completes the 
proof. □ 

Our assumption about the rotation set allows us to prove more: 
Claim 6. tt(u; + ) n tt(u;_) ^ 0. 

Proof. Suppose for a contradiction that tt(oj+) (1 7r(u;_) = 0. Note that by our 
assumptions, f{z\) = Z\ + (0,1) and w(zi) G Ess(/). Fix e < 1/4 such that 
B = B e (z 1 ) satisfies f(B) n £> = 0. By the previous claim, 7r(_B) intersects tt(oj + ) 
and 7r(w_), which means that there exist Pi,P2 € Z such that w_ + (pi,0) and 
w + + (p2,0) both intersect B. If £q C B is a straight line segment (not including 
its endpoints) joining a point of ui- + (pi,0) to a point of u + + (p2 7 0), then one 
can find a sub-segment I C £q that has the same property as Iq but in addition is 
disjoint from w_ + (px, 0) and w+ + (f>2i 0). 

This means that the hypotheses of |KT12b[ Proposition 4.10] hold. In particular, 
part (5) of said proposition implies that one of the following options holds: 

(1) For each x ^ (u>_ + (pi, 0)) U (cj + + (p 2 , 0)), there is m G 7L and S > such 
that f k (x) is disjoint from i?a(a; + (0, m)) whenever m < mo and /c > 0, or 

(2) A similar property as (1), with k < instead of k > 0. 

Suppose first that (1) holds. Note that since 7r(w + ) n 7r(cj_) = 0, the point tt(2_i) 
is disjoint from either 7r(w + ) or 7r(w_). Suppose n(z—i) £ 7r(w + ) (the other case 
is analogous). Choose z'_ x € 7r _1 (7r(,z_i)) (i.e. an integer translate of z_i) such 
that the first coordinate of z'_ l is greater than p\. This choice guarantees that 
z'_i £ cj_ + (pi,0), and since n(z'_ 1 ) ^ 7r(o;+) we also have that z'_ 1 £ u> + + (p 2 ,0). 
Thus, setting x = z'_ Y in (1) we conclude that is disjoint from -B^rr + (0, m)) 

whenever m < mo and k > 0. This contradicts the fact that f^(x) — x — (0, fc) for 
all fc G N (since x is an integer translate of 

If (2) holds instead, a similar argument using z\ instead of Z-\ leads to a con- 
tradiction. Since all cases lead to a contradiction, the proof is complete. □ 

3.2. Chains of disks with a special property. Note that by Claim |6j there 
exists mo such that {uj + — (mo, 0)) fl w_ ^ 0, and so a similar property holds if one 
replaces mo by any m > mo- Let 

F m := (oj + - (m,0)) U cj_, 

and given z G M 2 \ F m denote by O m (z) the connected component of R 2 \ F m 
containing z. Recall from |KT12b[ Proposition 4.11] the following facts: 

• O m (z) is an open topological disk for any m G N, and 

• O m (z) n (O m (z) + (0, k)) = for any integers k ^ and m > mo- 
Note that (F'm)meN is an increasing chain of sets, so (O m (-?))meN is a decreasing 
chain of disks. 

Lemma 3.4. For any z G T 2 \ 7t(oj + ) there exist w G Z 2 and z G 7r _1 (z) smc/i £/ia£ 
(1) Z F m for any m G N, and 
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(2) for any v G Z 2 not parallel to w, there is m > too such that 

O m {z)f\{O m {z) + v) = %. 

Proof. Choose z G n^ 1 (z) such that Pi(z) > 0. Since Pi(w_) C (— oo,0], it follows 
that z £ Moreover, z ^ u + — (m, 0) for any to € Z since z = 7r(z) ^ 7r(o; + ) by 
hypothesis. In particular, z ^ _F m for any m G N. 

For m > mo, let O m = O m (z). As we already mentioned, by [K T12b[ Proposi- 
tion 4.11] we have that O m is an open topological disk and O m n (O m + (0, k)) = 
for all integers k ^ 0. Suppose for a contradiction that (2) does not hold. Then 
there exist two non-parallel vectors u, v in Z 2 such that O m (1 (O m + u) ^ and 
O m H (O m +v) for all m G N. If u = (tti, U2) and t? = («i, U2), then the previous 
remarks imply that u\ 7^ and i>i 7^ 0. We may assume without loss of generality 
that u\ > and V\ > (otherwise we replace u by — u or u by —v as necessary). 

Let 

W m =\JO m +iuLl\JO m + jv. 

Note that, by our assumption, W m is connected. Moreover, W m contains z + v%u 
and z + u%v, and so W m intersects W m + {v\U — Uxv) = W m + (0, k), where k = 
V1U2 — u\V2 G Z. The fact that u and v are not parallel implies that k ^ 0. 

On the other hand, since O m is disjoint from F m , we have that for i € N the set 
O m + iu is disjoint from 

F m + iu — F m + (mi, 0) = [u>- + (mi, 0)) U (u + + {iu\ — to, 0)), 

where we used the fact that F m + (0, n) = F m for any n G Z. Moreover, since 
w_ C w_ + (mi,0) (because iu\ > 0), we conclude that O rn + iu is also disjoint 
from F m - iUl = w_ U (cj + + (m x - to, 0)). 

In particular, if < i < v 1, we have that w + + (vi^i — to, 0) C w + + (mi — to, 0), 
so that -Fm-^u! C F m -i Ul . Hence O m + iu is also disjoint from F m - VlUl . A similar 
argument shows that O m + jv is disjoint from F m - VlUl if < J <u\. 

Thus we conclude that W m CK 2 \ F m - VlUl . In particular, using to = too + v\U\ 
we see that W m CM 2 \ F mo . Since W m is connected and contains 2, the definition 
of O mo implies that W m C O mo . But we have shown that W m intersects W m + 
(0,k) where k = viu 2 — u\v 2 7^ 0, while O mo is disjoint from O mo + (0,k). This 
contradiction proves our claim. □ 



Using the terminology of |KT12b| Sec. 3], the Lemma 3.4 says that (O m (2)) m£ N 
is an eventually Z 2 \ Kw-free chain of open topological disks. 

3.3. Localizing the points Zj in lu + . Let us state a general fact |KT12bl Propo- 
sition 3.3]. The only hypothesis used in this proposition is that / is a lift of a 
homeomorphism nomotopic to the identity. 

Proposition 3.5. Suppose that for some w G Z 2 there exists an eventually Z 2 \Rw- 
free decreasing chain of open connected sets (O m )meN such that f(O m ) = O m and 
i(O m ) is fully essential for each to G N. Then f has uniformly bounded deviations 
in some direction, i.e. there exists w G M 2 and M > such that 

\Pw{P(z) -z)\< M for all n el, ze R 2 . 

In our current setting, this allows us to obtain the following: 
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Claim 7. Tv(zi) € 7r(w+) for each i E {— 1, 0, 1}. 



Proof. Fix i e { — 1, 0, 1}. We will assume that Zi is fixed for /. Otherwise, we may 
replace / by the new lift / — (0, i) of /, which fixes Zi, and proceed with the same 
proof (note that this leaves the sets uj± unchanged). 

Assume for contradiction that ir(zi) £ it(uj+). Then Lemma 3.4 implies that 
there is an integer translate z[ of Zi and some w £ Z 2 such that z[ ^ F m for all 
m G N and (O m (^))meN is an eventually 1? \ wZ-free chain of disks. Moreover, as 
we already saw, Tr(z' i ) € Ess(/), so that Tr(O m (z' i )) is fully essential, for each m € N. 
In addition, O m {z'^) is invariant, since it is a connected component of the comple- 
ment of an inva riant set containing z[, which is fixed by /. Thus the hypotheses of 
hold, and we conclude that / has uniformly bounded deviations in 



3.5 



Proposition 

some direction. This is not possible due to the fact that p(f) is a nondegenerate 
vertical interval (which implies unbounded deviations in all non-horizontal direc- 
tions) and by our assumption the deviations in the horizontal direction are not 
uniformly bounded. This contradiction proves the claim. 

□ 

3.4. The dynamics on the cylinder. Denote by Ti,T 2 the translations (x,y) H > 
(x + l,y) and (x, y) H ^ (x,y + 1), respectively. We will consider the cylinder 
A = M 2 /(T 2 ) ~ K x T 1 . Let r: R 2 -> A be the covering projection. The covering 
7r : K 2 — > T 2 also induces a covering tt : A — > T 2 naturally, and the lift / induces a 
lift / of / such that the following diagram commutes. 




Consider the set ui+ — t(cj+). Then a;+ is /-invariant, and every connected com- 
ponent of uj + is unbounded to the right (and bounded to the left) in A. Moreover, 
the fact that w + is closed and T 2 -invariant implies that ui + is closed in A. 

Let A* = A LI {Loo} the space obtained by adding to A one of its topological ends 
(the one on the left, L^) and topologized so that A* ~ M 2 . This is formally done 
by letting represent an arbitrary new point, and defining an open set of A* to 
be any union of open sets of A with sets of the form {(x, y) G A : x < a} U {L^} 
for some ael. 

Being homotopic to the identity, the map / leaves the two ends of A invariant, 
and in particular / extends to a map /* of A* by fixing the point L^. Let Z7* be 
the connected component of A* \ uJ^. which contains L^. Then [/* is an open 
invariant set, and is also simply connected (since A* ~ M. 2 and every connected 
component of the complement of is unbounded) . 

Moreover, U* itself is unbounded in A*. Indeed, in the case that £/* is bounded, 
it follows that U = \ {L^} is bounded to the right in A, and so t _1 (J7) is 
bounded to the right in K 2 . Thus, if H~ denotes the half-plane {(x,y) : x < 0} 
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in K 2 , then U = r _1 (£/) is an open /-invariant set such that U C H~ + (m, 0) 
for some m £ N. Furthermore, H~ — (1,0) C U, due to the definition of w+ 
and J7*. Thus H~ — (1,0) C U C H~ + (m,0), and this easily implies that / 
has uniformly bounded displacement in the horizontal direction (see for instance 
|KT12c( Proposition 1.5]), which is not possible under our assumptions. 

Thus /» is a homeomorphism of a plane, preserving orientation (since / is homo- 
topic to the identity) and leaving and unbounded open topological disk [/* invariant. 

Note that in the previous section we showed that ^{zi) € 7t(ll> + ), so in particular 
replacing Zi by some integer translate we may assume that zi G lo + for each i G 
{ — 1,0, 1}. This means that if 2j = 7r(zj), then i; e A, \ {/*. Recall that f(zi) = 

+ (0, i), so that the rotation number of the fi xed point z% of / relative to the lift 



/ is i (see the comments before Corollary 2.7 observing that the cylinder A used 
there is vertical instead of horizontals 



We would like to apply Corollary 2.6 to /* and U*. To meet the hypotheses of 



the corollary, we first need to verify that /* is nonwandering (at least in [/*). 
Claim 8. / is nonwandering in A. 

Proof. Recall that we are assuming that / preserves a Borel probability measure 
of full support fj, on T 2 . There is a corresponding lifted (non-finite) /-invariant 
measure fl on IR 2 which is invariant by Z 2 -translations, positive on open sets, and 
finite on compact sets (ju satisfies j2(E) — fi{%(E)) for any Borel set E such that 
tt\e is injective). This induces a measure ju on A which is also /-invariant, finite on 
compact sets, positive on open sets and Ti-invariant. 

Suppose there is an open set V G A which is wandering for /. By choosing a 
relatively compact open subset of V, we may assume that J1(V) < oo. Moreover, we 
may assume that V C Q, where Q = {(x, y) € A : < x < 1}, by using a suitable 
Ti-translation of V intersected with Q. Letting Ck = {(x, y) G A : — k < x < k}, we 
have that Jl(Ck) — 2kp{Q). On the other hand, the /x-measure of V n = U2=i f % (Y) 
is riji(y). 

Let r = Jj,(V) I '(2]Z(Q)) , and let k n be the integer such that nr — 1 < k n < nr. 
Then Jl(Ck n ) < fi(V n ), and so there is a point of V n in the complement of Ck n - This 
means that there exists z n G V and an integer 1 < i n < n such that f tn (z n ) C^ nl 
which implies that 

\Pl{f n {Zn))-Pl(z n )\>k n -l, 

where Pi : A — > M is the projection onto the first coordinate. 
Letting z n be an element of ■n^ 1 (z n ) in [0, l] 2 , we see that 



Pi 



f n ( Zn ) Z n 



k n — 1 nr — 2 
> > > r. 



Choosing subsequences, we may assume that (f ln (z n ) — z n )/i n converges to some 
sei 2 , which by definition must be in p(f). Our previous observation implies that 
the first coordinate of v has modulus at least r. In particular, the first coordinate 
of v is nonzero, contradicting the fact that p(f) is a vertical interval containing the 
origin. □ 
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The previous claim i mpl ies that /* is nonwandering in [/*, so we are under the 
hypotheses of Corollary \2.7\ using /* as the map /, and as zq. Since we already 
saw that [/* is unbounded, we conclude that all periodic points of /* in A* \ U* 
have the same rotation number relative to the lift /. This contradicts the fact that 
z and Z\ are fixed points of / in ui + C A* \ U* with respective rotation numbers 
and 1. This contradiction completes the proof of Proposition |3.1| □ 



Remark 3.6. The only point of the proof where the existence of an invariant measure 
of full support is essentially used is in the proof of Claim [8] For all other proofs in 
this article it suffices to assume that / is nonwandering. 

4. A REALIZATION THEOREM FOR FULLY ESSENTIAL INVARIANT CONTINUA 

This result improves Theorem D of [KT12c . 

Theorem 4.1. Let f be a nonwandering homeomorphism homotopic to the identity, 
f a lift of f to M. 2 , and K C T 2 a fully essential f -invariant continuum. Given 
seZ 2 and q G N, if there exists z G M 2 such that f q {z') — z' + v, then there also 
exists z' G M. 2 such that n(z') G K and f q (z') = z' + v. 

Proof. Let g — f q — v, so that g is a lift of g — f q and by hypothesis g has at 
least one fixed point. We would like to show that g has a fixed point z' such that 
ir(z') G K. 

Fix m G N so large that the only element of mZ 2 n p(g) — {(0, 0)} (the origin 
must belong to p(g) since g has a fixed point). Consider the map g induced by 
g on the torus f 2 = R 2 /(mZ 2 ). Denote by t: K 2 R 2 /(mZ 2 ) the quotient 
projection. There is a finite covering tt : T 2 — >• T 2 defined by the projection T 2 — > 
f 2 /(Z 2 /mZ 2 ) ~ T 2 , and g is a lift of g by w. 

Note that z G M 2 is a fixed point of <? if and only if t(z) is a fixed point of 
g. Thus, to prove the theorem it suffices to show that g has a fixed point in the 
compact ^-invariant set K — n (K). 

Observe that K is a fully essential continuum in T 2 . Indeed, suppose that some 
connected component U of T 2 \ K is essential in T 2 . Since the map induced by 7? 
in the fundamental groups is injective (explicitly, it is the inclusion mZ 2 — > Z ), it 
follows that tt(U) is connected and essential in T 2 . But since tt(U) is disjoint from 
K, this contradicts the fact that K is fully essential. 

We need to show that K contains a fixed point of Ij. Suppose for a contradiction 
that K n Fix(g) = 0. Then the connected components of T 2 \ K (which are open 
topological disks) form an open covering of the compact set Fix(t/), and so we can 
find finitely many such components U\, . . . , t/fc such that Fix(g) C U\ U • • • U 
Moreover, we may chose them so that each Ui intersects Fix(gf), and in particular 
each Ui is ^-invariant. 



Note that 'g is nonwandering due to Proposition 2.5 and dUi C K has no fixed 
points of g. These two facts and a classic result of Cartwright and Littlewood 
allow to conclude that the extension of g\ut to the prime ends compactification 
of Ui has no fixed points in the circle of prime ends, and therefore there exists a 
closed topological disk Di C Ui such that Fix(gf) n Ui C A and the fixed point 
index of g in Di is 1 (this argument is contained in Proposition 4.2 of |Korl0j ). 
Thus, the Lefschetz index of g is exactly k. Since by our hypothesis g (and so g) 
has at least one fixed point, k > 1. But on the other hand the Lefschetz index 
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of a homeomorphism of T 2 homotopic to the identity must be 0, so we have a 
contradiction. □ 

5. Existence of an essential invariant annulus 

In this section we prove a result that, together with Proposition |3.1| completes 
the proof of Theorem [XJ Note that it does not require the existence of an invariant 
probability measure of full support. 

Theorem 5.1. Let / : T 2 — > T 2 be a nonwandering homeomorphism homotopic to 
the identity and f a lift of f. Suppose that there exists M > such that 

|Pi(/"(z) -z)\< M for all n el, z£ M 2 , 

and p(f) is not a single point. Then there exists an invariant vertical annulus. 

We begin with a simple lemma. 

Lemma 5.2. Suppose that a homeomorphism f : T 2 — > T 2 homotopic to the identity 
is nonwandering and has an invariant compact set K which is essential but not fully 
essential. Then f has a periodic essential annulus A which is cither invariant or 
disjoint from its image. In addition, if A is vertical and some lift f of f is such 
that p(f) C {0} x R, then f(A) = A. 

Proof. The hypothesis implies that there exists a connected component Aq of T 2 \K 
which is neither essential nor fully essential. Since the connected components of 
T 2 \ K are permuted, Aq is /"-invariant for some n £ N. Moreover, either n = 1 or 
f{A ) n A = 0. Let A be the "filling" of A, i.e. the union of A with all inessential 
connected components of T 2 \ Ao. It is easy to verify that A is homeomorphic to an 
essential open topological annulus. Moreover, it is still true that f n {A) = A and 
cither n = 1 or f(A) n A ^ 0, so the first claim follows. 

To prove the second claim suppose for a contradiction that n > 1, so that 
A is disjoint from its image. If A is a connected component of 7r _1 (A), then 
A = A + (0, 1) and A is disjoint form A + (fc,0) for any 6 Z,. Moreover, there 
is k G Z such that f n (A) = A+ (k,0). Since Pi (/>(/)) = and P\{A) is bounded, 
one easily sees that k = 0; thus f n (A) = A. 

On the other hand, since f(A) n A = 0, one also has f(A) n A = 0. Note 
that exactly two connected components of M 2 \ A are invariant by the translation 
T2(x,y) = (x, y + 1). One of these components is unbounded to the right, say 
W+, and the remaining one, W-, is unbounded to the left. Since f(A) is T2- 
invariant, we have either f(A) C W + or f(A) C W^. But the first case implies 
that f(W+) C W+, while the second case implies that /(W_) C W-. In cither 
case, one concludes f n {A) fl A = 0, a contradiction. □ 

Proof of Theorem \5.1\ The proof is somewhat similar to the proof Proposition |3.1| 
Let H = {(x,y) : x < 0}, and let W be the unique connected component of 

Unez f n (H) which is unbounded to the left, so that H c W C H + (m, 0) for some 

m £ N, due to our hypothesis. 

Note that W is /-invariant and W + (0, 1) = W. This implies that K = tt(5 W) 

is compact and /-invariant. Clearly K is essential, since it intersects any horizontal 

loop in T 2 (indeed, if 7 is a horizontal loop in T 2 then any connected component of 
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7r _1 (7) is a connected set intersecting both W and its complement, and therefore 
intersecting dW). 

Suppose that K is not fully essential. Since K is ess entia l and invariant, and 
clearly under our assumption P\(p(f)) — {0}, by Lemma 5.2 there is an invariant 



essential annulus A. Using the assumption that p(f) is not a single point, so that 
p(f) — {0} x [a, 6] for some a < b, one easily concludes that A is a vertical invariant 
annulus, as sought. 

It remains to consider the case where K is fully essential. We will show that 



this case is not possible. To see this, first observe that from Theorem 4.1 and 
the fact that /?(/) necessarily contains different rational points, there are points 
zo,zi € K 2 such that n(zi) £ K, integers p$,pi and positive integers qo,qi such 
that pi/qi 7^ Po/qo and f qi (z^ — Zi +Pi- We may assume that Zi £ dW = 7r _1 (.K") 
by using an appropriate integer translation of z%. 

Let A = M 2 /(r 2 ) ~ KxT 1 , where T 2 (x, y) = (x,y+l), and denote by t: R 2 -> A 
the projection. Let / be the map induced by / on A, and T\{x, y) — (x + 1, y) the 
horizontal translation. 

We claim that / is nonwandering. To see this, suppose that U is a wandering 
open set of /. By using an appropriate Ti-translation of U and choosing a smaller 
set, we may assume that U C [0, 1] x T 1 . Choose an integer m > 2M + 1, and note 
that f 2 = (R/mZ) x T 1 ~ A/(ff-) is a finite covering of T 2 , while it is covered by 
A after natural identifications. Let a : A — > T 2 be the projection, and let F denote 
the map of T 2 lifted by /. Then F is a lift of / to the finite covering T 2 , and since / 
is nonwandering, By Proposition |2.5| we know that F is nonwandering. Note that, 
since m > 2M + 1, the projection a is injective on [— M, M + 1] x T 1 . But our 
hypothesis on the deviation of / implies that f n (U) C \—M, M + 1] for all n E Z, 
and so the fact that o"|[_m,m+i] i s injective implies that 

F n (a(U)) n a(U) = a(f n (U)) H a(U) - <j(f n {U) f\U) = % 

for all n £ 1. Hence <j(U) is a nonempty wandering open set for F, a contradiction. 

Finally, we observe that W = t(W) is an open subset of A such that (— oo, 0) x 
T 1 C W, and since W is simp ly connected and T2-invariant, W is a topological 



annulus. Thus, as done in ^3.4 if A* = A U {Loo} is the plane obtained by adding 
to A its end on the left side, W* = VFLl{L 00 } is a topological disk which is invariant 
by the map induced by / fixing (which is still nonwandering), and % :— r(zi) 
is a periodic point of / in the boundary of W*, for i £ {0, 1}. But since the points 
Zi have different rotation vectors associated to the lift /, this contradicts Corollary 



2.7[ completing the proof. □ 

6. Proofs of Theorems 1X1 and IbI 
Theorem \K\ follows immediately from Proposition |3.1| and Theorem |5.1| 



6.1. Proof of Theorem |B| Let / and / be as in the hypotheses of the theorem, 
and suppose that p(f) is a segment of r ation al slope containing a point of rational 



coordinates. By the properties stated in f 2.1 there is n £ N such that p( f n ) contains 



a point of Z 2 , and there is A £ GL(2, Z) such that p{Af n A^ 1 ) = Ap(f) is a vertical 
segment, which also contains a point v £ Z 2 . Letting g = Af n A~ 1 ~ v, we see that 



16 



NANCY GUELMAN, ANDRES KOROPECKI, AND FABIO ARMANDO TAL 



g is a lift of g = Aj2 f n AZ£ (where Aj2 is the map induced by 4 on T 2 ) and the 
hypotheses of Theorem [X] hold for g. Thus there is an invariant vertical annulus U 
for g, which means that Aj2U is a periodic essential annulus for /. 

The converse direction of the theorem is an easy consequence of the definition of 
rotation set. For completeness, suppose that A is a periodic essential annulus and 
let k £ N be such that f k {A) = A. By items (2) and (3) of [KT12c( Proposition 
1.4], there is v £ Z 2 and some lift g of f k to R 2 such that p(g) C Rv (in fact v is the 
homological direction of A) . Since is another lift of f k , one has that g = + w 



for some w £ Z , and by the properties from [2.1 one has 



kp(f) +w = p(f k +w) = p{g) C Rv, 

which implies that p(f) C Rv — w/k. Since p(/) is assumed to have more than one 
point, it must be a segment of rational slope parallel to v. Since Rv — w/k contains 
w/k £ Q 2 , it follows that Kt>nQ 2 is dense in Rv, and in particular p(f)riQ 2 ^ 0. □ 
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